47 research outputs found
Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics
A continuous time model for multiagent systems governed by reinforcement
learning with scale-free memory is developed. The agents are assumed to act
independently of one another in optimizing their choice of possible actions via
trial-and-error search. To gain awareness about the action value the agents
accumulate in their memory the rewards obtained from taking a specific action
at each moment of time. The contribution of the rewards in the past to the
agent current perception of action value is described by an integral operator
with a power-law kernel. Finally a fractional differential equation governing
the system dynamics is obtained. The agents are considered to interact with one
another implicitly via the reward of one agent depending on the choice of the
other agents. The pairwise interaction model is adopted to describe this
effect. As a specific example of systems with non-transitive interactions, a
two agent and three agent systems of the rock-paper-scissors type are analyzed
in detail, including the stability analysis and numerical simulation.
Scale-free memory is demonstrated to cause complex dynamics of the systems at
hand. In particular, it is shown that there can be simultaneously two modes of
the system instability undergoing subcritical and supercritical bifurcation,
with the latter one exhibiting anomalous oscillations with the amplitude and
period growing with time. Besides, the instability onset via this supercritical
mode may be regarded as "altruism self-organization". For the three agent
system the instability dynamics is found to be rather irregular and can be
composed of alternate fragments of oscillations different in their properties.Comment: 17 pages, 7 figur
Size Doesn't Matter: Towards a More Inclusive Philosophy of Biology
notes: As the primary author, O’Malley drafted the paper, and gathered and analysed data (scientific papers and talks). Conceptual analysis was conducted by both authors.publication-status: Publishedtypes: ArticlePhilosophers of biology, along with everyone else, generally perceive life to fall into two broad categories, the microbes and macrobes, and then pay most of their attention to the latter. ‘Macrobe’ is the word we propose for larger life forms, and we use it as part of an argument for microbial equality. We suggest that taking more notice of microbes – the dominant life form on the planet, both now and throughout evolutionary history – will transform some of the philosophy of biology’s standard ideas on ontology, evolution, taxonomy and biodiversity. We set out a number of recent developments in microbiology – including biofilm formation, chemotaxis, quorum sensing and gene transfer – that highlight microbial capacities for cooperation and communication and break down conventional thinking that microbes are solely or primarily single-celled organisms. These insights also bring new perspectives to the levels of selection debate, as well as to discussions of the evolution and nature of multicellularity, and to neo-Darwinian understandings of evolutionary mechanisms. We show how these revisions lead to further complications for microbial classification and the philosophies of systematics and biodiversity. Incorporating microbial insights into the philosophy of biology will challenge many of its assumptions, but also give greater scope and depth to its investigations
Mitochondria as integrators of information in an early-evolving animal: insights from a triterpenoid metabolite
Mitochondria have the capacity to integrate environmental signals and, in animals with active stem cell populations, trigger responses in terms of growth and growth form. Colonial hydroids, which consist of feeding polyps connected by tube-like stolons, were treated with avicins, triterpenoid electrophiles whose anti-cancer properties in human cells are mediated in part by mitochondria. In treated hydroids, both oxygen uptake and mitochondrial reactive oxygen species were diminished relative to controls, similar to that observed in human cells exposed to avicins. While untreated colonies exhibit more stolon branches and connections in the centre of the colony than at the periphery, treated colonies exhibit the opposite: fewer stolon branches in the centre of the colony than at the periphery. The resulting growth form suggests an inversion of the normal pattern of colony development mediated by mitochondrial and redox-related perturbations. An as-yet-uncharacterized gradient within the colony may determine the ultimate phenotypic effect of avicin perturbation